When the Higgs meets the Top: Search for t — >• ch° at the LHC 
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The Higgs boson h and the top quark t are the two most massive and newest particles ever 
discovered. If f — >■ ch'^ occurs at the level of a percent or so, the observed ZZ* and 77 signal events 
may have accompanying cbW activity coming from tt feeddown. A general two Higgs doublet model 
brings in new ct, cc and tt couplings that modify properties of the light Higgs h'^, and t -^ ch'^ can 
be searched for via h" — >■ ZZ", 77, WW* and bb (even t'^t~) modes in ft events. Existing data 
might be able to show some clues for t — >■ c/i" signature, or should provide important information 
for B{t — )■ ch") down to below the percent level. 
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Introduction and Motivation 

With the landmark discovery [l|, 0] of a 126 GeV bo- 
son in 2012 by the ATLAS and CMS experiments at the 
Large Hadron Collider (LHC), efforts have shifted to- 
wards Higgs property studies, to confirm that this is in- 
deed the Higgs boson of the Standard Model (SM), or to 
find deviations indicating that it may not be the Higgs 
boson. In this connection, we stress that the Higgs boson 
and the top quark are the two most massive, as well as 
newest, particles ever discovered. With the Higgs boson 
as the "mass giver", it is natural to ask whether it re- 
veals any special effects associated with the heaviness of 
the top quark. A related question is, analogous to the 
generation repetition seen with fermions, if we have seen 
one Higgs boson, could there be others. 

We wish to explore a possible tch'^ coupling, where h!^ 
stands for the 126 GeV boson. Absent within SM, this 
coupling has not been the focus of attention. At the one 
loop level, the SM branching ratio (for to/jO = 120 GeV) 
B{t — ^ ch^) ~ 4.6 X 10^^** [3|] is extremely suppressed. 
But with TO/jO < nit, t ^ ch^ decay [3-0| can readily be 
searched for in ti production. In the post-discovery era, 
we must search for t —>■ chP at the LHC as part of the 
Higgs, and top, property programs. 

The B — )■ D^*^Tiy "anomaly" uncovered by the BaBar 
experiment Q suggests the need of a multi-Higgs bo- 
son sector. The B — > D'^*^ti> rates are found to deviate 
from SM expectations, with combined significance over 
3cr. More intriguing, it cannot |8| be explained by the 
usual two Higgs doublet model (2HDM), type II, the 
popular Higgs extension realized in minimal supersym- 
metry (SUSY) . One explanation is to adopt Q a general 
2HDM, called 2HDM-III, allowing for flavor changing 
neutral Higgs (FCNH) couplings that modify the charged 
Higgs couplings, which could resolve the BaBar anomaly. 
The Natural Flavor Conservation (NFC) condition was 
proposed over 35 years ago jjQ] to forbid such couplings, 
but the FCNH cou pling s of Ref. go even beyond the 
Cheng-Sher ansatz [ll[, an approach designed to tame 
FCNH couplings involving lighter quarks. One basically 
lets data decide the strength of possible FCNH couplings, 
which cannot be argued against. Note that it was within 



the Cheng-Sher ansatz that t — >■ chP (or hP — >■ tc) decay 
was first proposed |4I| as the leading effect. 

To account for the BaBar anomaly, the FCNH coupling 
Pet of the exotic heavy Higgs doublet need to be of order 1 
in strength [9| . For our purpose, keeping notation of the 
usual 2HDM-II [I4I, the observed boson hP may contain 
a small admixture cos(/3 — a) of the exotic neutral Higgs, 
hence the tchP coupling 13] 



Pet cos(/3 — a) cthP + h.c. 



(1) 



which can induce t — > ch^ decay. In Fig. 1 we illustrate 
the branching ratio B{t -^ ch^) vs pct cos{P — a). The 
question is: Considering all available data, what is the 
allowed B{t — )■ ch^), or equivalently, pct cos(/? — a) value? 
What are the signatures to pursue? 

We note that, if we take pct ~ 1, which is not quite ex- 
plored because it is suppressed by cos{f3 — a) for couplings 
involving h^ (but not suppressed for couplings involving 
the heavy exotic Higgs bosons H^, A^ and H^), then 
the analogous parameters ptt, Pcc, Pbb and prr will en- 
ter the Higgs property study program, as we shall eluci- 
date. An existing study of multi-lepton final states finds 
a bound ^ oi B{t ^ ch°) < 2.7%. But this should be 
taken with some caution, as it assumed SM branching 
ratios for h° ^ WW*, ZZ* and tt final states. The 
study took an effective field theory approach to isolate 
the tch'^ coupling, hence is not a full theory. 



0.1- 



T 0.05 



0.1 0.2 0.3 0.4 0.5 0.6 

|prtCOS(^-ff)| 

FIG. 1. B{t -> ch°) vs Pct cos(;3 - a), with 2% indicated. 
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FIG. 2. Allowed prr-p„t region for 2HDM-III to solve 
the B — >■ D^*'tu anomaly, taking 171^+ = 700 GeV (and 
Pbb ~ 0). The shaded-green area is the combined result from 
TZ{D) (solid-blue lines) and TZ{D*) (dashed-red lines), while 
the dotted-purple lines illustrate the /i" — >■ tt bound by tak- 
ing c^_c, = 0.2 in Eq. ®. 



Implications oi B ^ D^*^ti' Anomaly and b 



sj 



The BaBar experiment measured the ratios TZ{D^*^) — 
T{B -^ D^*'^Tv)/T{B -^ D^*kiy), finding them both 
larger than SM expectations, with a combined signif- 
icance of 3.4(7. In 2HDM-II, this implied d] that 
tan^/mH+ = 0.44 ±0.02 GeV"! and 0.75 ±0.04 GeV"! 
from TZ{D) and TZ{D*), respectively. The impressive pre- 
cision is because many uncertainties, both from measure- 
ment and from theory, cancel. The two numbers are in- 
compatible with each other, hence "excludes the type 
II 2HDM charged Higgs boson with a 99.8% confidence 
level for any value of tan (3/mH+ " |8J . Either tan (3/mH+ 
value, however, would over-enhance |14| B — >■ ti/, which 
is found in agreement (in order of magnitude) with SM 
expectation, spelling further trouble. 

Employing FCNH parameters in 2HDM-III to ac- 
count [9] for the BaBar "anomaly" , a new c-t coupling 
(and u-t for B -^ tv), heretofore forbidden by the NFC 
condition |1G| . needs to be of order 1. For the lepton 
sector, Ref. Q assumed the usual 2HDM-II coupling, i.e. 
Prr = — ta,Ti/3'\/2mr/v, where v is the weak scale. 

We shall use the following notation for the Yukawa 
couplings of the 2HDM-III Higgs sector [H], 



■ P^Cfj- 



u,d.e 

+ («/c^-„ - p^sp^^) iJ° - isgn(Q/)/75A°] / 
- [u {Vp'^R - p'^VL) dH+ + vp'^mH+ + h.c] , (2) 

where sp-a {cp-a) stands for sin(/3 — a) (cos(/3 — a)) in 
the 2IIDM-II notation T2] for sake of comparison (even 
though (3 is no longer physical) . The diagonal k terms re- 
late to mass generation, while the hermitian Il3| p terms 
come from the second Higgs doublet, and can have off- 
diagonal terms allowed by data. It is the combined effect 
of Pet and Prr, both entering through the H^ couplings, 
that can account for B -^ D''*^tv. Stringent con- 
straints from down quark sector imply that only /9f,b needs 
to be considered; note that the superscript for p can be 



FIG. 3. Constraint on pi,b from b -^ 37, assuming pct = 1, 
Pec = 0.2 and m^+ = 700 GeV. 



dropped for flavor-specific elements. We shall only keep 
Ptt from lepton sector, but it is pct, Ptt and pcc from up 
quark sector that are of interest. 

In the so-called "decoupling limit" [13, [la|, cos(/3 — 
a) — > and sin(/3 — a) — >■ 1, ft,° becomes just the Higgs 
boson of SM, while the heavy H'^, A^ and H^ form an 
exotic heavy scalar doublet with FCNH couplings. This 
limit was tacitly assumed in Ref. ^2], which advocated 
i/°, A'^ -> tc search. But we entertain finite c^_q values 
for sake of i — >■ c/i° decay, hence would need to consider 
/i° — >■ TT data. The latter from vector boson fusion pro- 
duction is within a factor of 2 from SM expectations. 



|S/3- 



{prrV / V2mr)ci3-a\ < v2- 



(3) 



With Ptt — —0.5 (9J, even a small c^-q could upset this 
bound. To illustrate further, we plot in Fig. 2 the range 
for p^^-pi-i allowed by BaBar anomaly for the typical 
value of mH+ = 700 GeV. The point prr — —0.5, pct '^ 1 
of Ref. [^, far outside the plot, would require cp-a to be 
rather small. If we take c^-_q — 0.2 (i.e. sp-a — 0.98) 
in Eq. dS]), then -0.12 < prr < 0.02 would push p^ to 
become very large, as seen from Fig. 2. 

Thus, ft.° — ?> TT data imply either one goes to the de- 
coupling limit of c^_a — > 0, where t — >■ ch'^ vanishes, or 
one has to entertain nonperturbative values for pd- As 
further analysis |l7| of q^ [tv pair mass) dependence of 
B — > Dtv favors New Physics from spin-1 particles, we 
shall detach from the BaBar anomaly, using it only to il- 
lustrate that Pct can be of order 1, and focus on probing 
tch° coupling directly at the LHC. 

But before turning to the LHC, we explore one other 
piece of B physics: 6 — >■ 37 (Fig. 3). We have already 
seen how hP — >■ tt data pushes down prr- We now show 
that, if one takes p^t ~ 1, the well-known b ^ sj pro- 
cess constrains pbb to be rather tiny (noted recently in 
Ref. Ts] in a different wa y) . 

In the notation of Ref. [l^ , the H^ loop gives 



SCr^i 



Ptt + TjrPct I I Pit + 



Ptt 



Vt. 



cb 



tb 



-Pct 



2m^/w2 



Pct Pbb 



2mtmii/v'^ ' 



(4) 



evaluated at matching scale ^w ^ Mw, where y — 
m^/M]j+ and F^y\y) are given in Ref. [H. The ef- 
fect through /9(,6 is enhanced by rrit/mi, as well as quark 
mixing elements, such that even a tiny pbb could affect 
& — >■ 57. We illustrate ptb vs pu in Fig. 3, where we take 
Pet = +1, mH+ = 700 GcV, and constrain B{B — > Xsj) 
to be within 50% of SM expectation. The "wrong-sign" 
Cj^ case has been included for comparison. Assuming 
Cj^ does not change sign, \pbb\ is constrained to be con- 
siderably less than 0.01. 

Probing tch° Coupling at LHC: General 2HDM-III 

With prr and pbb separately constrained to be small, 
we are still left with the up-sector exotic couplings, the 
FCNH Pet as well as the diagonal pcc and ptt, which we 
turn to constrain with present data. How large can B{t — >■ 
c/i°) be when pct ~ 1? I.e., what constraint do we have 
on cos(/3 — a)? We now take a direct search approach, 
namely, from knowledge of top quark physics. 

It is clear that Bch = B{t — > eft.") cannot be too large, 
for otherwise we should have seen deviations in tt mea- 
surements during the past two decades. The best mea- 
sured tt production cross section is a^^ ~ 162 pb by 
CMS [20| at \/s = 7 TeV via dileptons, with an experi- 
mental error of 5%. But although theoretical studies are 
approaching NNLO (next-to-next-to-leading order) QCD 
corrections, it is not yet complete. Comparing the two 
calculations of Refs. ^] and [22], theoretical errors could 
still be up to 10%. Bch up to ~ 5-6% seems still allowed, 
given that a^^ would be diluted by (1 — BchY- 

A multi-lepton analysis of Ref. [3 , based on 7 TeV data 
from CMS [23], gives a slightly more stringent bound 
of 2.7%, or [a ■ B](pp -^ tt ^ ch%W) < 9 pb (at 7 
TeV). We have commented that this study assumed SM 
branching ratios for hP decay, which should be taken with 
caution. However, it illustrates possible feeddown effects 
to observable Higgs boson decay modes, given the large tt 
production cross section at the LHC. Our chief suggestion 
is to inspect the clean ZZ* — >■ Ai samples of Higgs search 
data, which we now elaborate. 

What has been observed so far at the LHC is 



'gg- 



>/lO 



pSM 



pSM 
^ hO 



■B] 



SM 
ZZ' 



(5) 



where we assume h^ is produced dominantly through 
gluon-gluon fusion. We have separated respective pieces 
where /i°-properties may deviate from SM. Both exper- 
iments find consistency with the expected 15-20 £££'£' 
signal events expected from full 2011-2012 data set. How- 
ever, CT^t is of order 220 pb at 8 TeV [22] ■ If one takes 
Bch ^ 2.7%, this amounts to ~ 12 pb into ti -J> ch^bW, 
which should be compared with ^ 22 pb for gij-fusion 
production of a 126 GeV SM Higgs boson! An excess 
could have appeared already in ZZ* mode, except that 
each of the three product factors in Eq. ^ could devi- 
ate from SM. For example, (jgg^h° niay be smaller, or 
F/jO > T^^ might dilute direct production. 



TABLE I. Light Higgs h° properties in 2HDM-III with pc 



1. Widths are in MeV units, with F' 



ftO 



4.55 MeV. 



B- 



BM- 



nSM 



Comment 



WW 21.5% 0.98 hard to change sin(^ - a) ~ 1 

ZZ* 2.7% 0.12 hard to change sin(/3 - a) ~ 1 

77 0.24% 0.011 hard to change VF-loop dom. 



bh 

TT 
CC 



59.4% 2.70 hard to change 



5.7% 
2.6% 



0.26 
0.12 



within fac. 2 
up to ~ Pts 



b — > S7 

direct 

not measured 

jpcc < 0.2) 



99 



7.7% 0.35 



up to fac. 2 



pu 



1 



Compared with Ref. [3] , Eq. ^ allows us a more com- 
plete treatment of /i°-properties with pct ^ 1, hence un- 
derstand what SM-like observation of ZZ* may imply. 
Our study also illustrates how 2HDM-III with FCNH 
could alter several Higgs properties, driving in the im- 
portance of their measurement. 

With ft." dominantly the SM Higgs boson, its WW* 
and ZZ* decay rates, proportional to sin (/3 — a), are 
hardly changed. Likewise, the ft" — >■ 77 rate, dominated 
by IF-loop, is also SM-like. For fermions, the mass gener- 
ating K terms are close to SM, while a small cos(/3— a) (we 
are close to decoupling limit) dilutes the effect of p-type 
couplings. The consistency of ft" -^ tt with SM con- 
strains prr to be small, while pbb is constrained by 6 — )■ 57 
to be tiny if Pct ^ ^- Further diluted by cos(/3 — a), the 
bb rate arises from Kbb and is SM-like. 

We are left with potential pcc and ptt effects. The cc 
mode is extremely hard to search for, hence there are no 
limits so far. With cos(,3-a) -- 0.2, pcc ^ 0.2 [24] would 
bring Fee ^ ^bb — ^hh^^ ^^'^ t^'^ enhanced F^o would di- 
lute the Higgs signal. This can be partially compensated 
for by Ptt , as this parameter should naturally be of order 
1 if Pet '^^ 1, since Ktt — 1 also. With some suppression 
by cos(/3 — a), nevertheless it could bring Ogg^h" up or 
down by a factor of ~ 2. 

We summarize in Table I possible effects of our con- 



strained 2HDM-III (with Pet -I)- While F^o^^Z'/r^^ 



is similar to B' 



SM ^ 
ZZ*' "gg^h« 



could change by a factor of 2 



and F^Q could be enhanced. We comment: 

• If Pcc is small and ft" branching ratios are SM-like 
(except for gg mode), then the bound oi B{t — > 
eft") < 2.7% from Ref. would apply; 

• For enhanced agg^^o , then dilution of Bzz* would 
be necessary, implying enhanced ft" -^ cc; 

• If Ggg^j^o is suppressed, or Bzz* is diluted, then 
more ZZ* events may come from tt feeddown! 

Given the clean signature of the ZZ* or Ul'i' mode, 
the searches at CMS and ATLAS have been carried out 
in an inclusive way, i.e. simply reconstruct four charged 
leptons without looking into any associated byproducts. 
The experimental results can thus be used to constrain 
any other Higgs production process by looking into the 



extra activities in the events, which aheady have a clean 
peak around 126 GeV. There may weh be some fraction 
of Ue.'i' + cbW events, with up to 4 jets. 

The CMS prehminary resuh with fuU 7 and 8 TeV 
data [25 shows 13, 8, and 4 events with 0, 1, and 
2 jets, respectively, after selecting events with mu G 
(121.5, 130.5) GeV. There is no indication for higher as- 
sociated jet activity. To extract a bound on B{t — ?► chP), 



we assume a 



gg- 



>/lO 



Bih" 



ZZ*) takes SM value. By 



inserting the CMS data points, together with the back- 
ground histograms provided in the same plot 25|, and 
jet multiplicity distribution from top events, an upper 
limit on the top-Higgs contribution is estimated based 
on the standard CLs method [2al used at the LHC. The 
resulting 95% confidence level limit on the relative signal 
strength between t — > diP and inclusive Higgs production 
is around 31%, which can be converted to a limit of 6.5 
pb on the effective cross section of i — ?> eft." at 8 TeV, or a 
branching ratio limit around 1.5%. A genuine analysis is 
best left to the experiments, as data is already at hand. 
Our argument can be applied to the other mode, 77, 
that drives the Higgs boson discovery. But this mode is 
not so clean, and it is not clear how far one can go. Be- 
sides the aforementioned multi-lepton analysis, which is 
in fact driven by the /i° — >■ WW* final state, there is one 
other "steadfast" analysis one could do, which is search- 
ing for h^ -^ bb mode in ti — ^ ch^bW — > cbbb + iv. It has 
been shown [y] that, through heavy use of 5-tagging and 
mass reconstruction, one should be able to push down to 
1% sensitivity with 2011-2012 data. Here, B(hP -)• bb) 



might get diluted by hP — > cc^ which was not considered 
in Ref. J6i], but there is hope that actual experimental 
analysis could do better than the theoretical study. 

It is assuring that, if /i" behaves SM-like except for 
inducing t — >■ ch^ decay, we have multiple methods to 
probe down to the 1% level. 

Conclusions 

It is of great interest to search for the link between 
the top quark t and the Higgs boson hP . A discovery of 
the t — >• chP process with present data would suggest the 
existence of an extended Higgs sector beyond the usual 
2HDM-II implied by minimal SUSY. It is rather impres- 
sive that the intense efforts of Higgs search in the past 
two years could already push the limit on t — > ch^ down 
to the percent level. Actual experimental studies, incor- 
porating also the ft." — >■ WW*, bb, t^t~ and 77 modes, 
should be able to push beyond what we have presented. 
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